Self-Aware Services: Using Bayesian Networks for Detecting Anomalies in Internet-Based Services

We propose a general architecture and implementation for the autonomous assessment of health of arbitrary service elements, as a necessary prerequisite to selfcontrol. We describe a health engine, the central component of our proposed ‘SelfAwareness and Control’ architecture. The health engine combines domain independent statistical analysis and probabilistic reasoning technology (Bayesian networks) with domain dependent measurement collection and evaluation methods. The resultant probabilistic assessment enables open, non-hierarchical communications about service element health. We demonstrate the validity of our approach using HP’s corporate email service and detecting email anomalies: mail loops and a virus attack. We also present the results of applying on-line machine learning to this architecture, and quantify the benefits of the Bayesian network layer.